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ABSTRACT 

A  statistical  analysis  of  the  Silas  B.  Hays  Hospital, 
North  Fort  Ord  Family  Practice  Clinic  was  conducted  over  a 
period  of  five  months  of  operation.   A  computer  simulation 
model  of  the  clinic  was  constructed.   The  response  of  the 
model's  statistical  measure  of  effectiveness  to  increased 
input  arrival  rates  and  the  addition  of  one  physicians' 
assistant  to  the  staff  was  measured  in  a  sensitivity  analy- 
sis.  Control  variate  and  antithetic  variance  reduction 
techniques  were  applied  to  the  model  and  compared  with  the 
ordinary  Monte  Carlo  method  for  their  effectiveness  in  the 
reduction  of  variance  of  the  output  statistic. 
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I.   INTRODUCTION 

A .   BACKGROUND 

A  recent  development  in  the  practice  of  medicine  has 
been  the  renewed  emphasis  on  a  continuous  and  personal  rela- 
tionship between  patient  and  doctor.   Manifestations  of  this 
trend  are  the  many  physicians  trained  in  the  new  family  prac- 
tice specialty,  serving  in  the  military  as  well  as  the  civil- 
ian sector.   The  principal  underlying  theme  of  this  new 
specialty  has  been  to  provide  continuous  care  to  patients 
within  a  family  doctor  framework.   In  fact,  the  Family  Prac- 
tice Physician  is  a  recognized  specialist  who  has  been  trained 
to  provide  total  family  care  usually  encompassing  acute  minor 
illnesses,  pediatrics,  obstetrics,  minor  surgery,  gynecology, 
internal  medicine  and  family  counseling.   Although  no  defin- 
itive statistics  are  available,  it  has  been  estimated  that 
approximately  80%  of  an  outpatient's  medical  demands  can  be 
met  by  a  family  practitioner. 

The  North  Fort  Ord  Family  Practice  Clinic,  hereafter  re- 
ferred to  as  FPC,  is  the  most  recent  clinic  of  its  type  to 
be  established  at  Fort  Ord,  California.   Its  operation  is 
very  similar  to  that  of  the  family  practice  clinic  located 
in  Silas  B.  Hayes  Hospital,  also  at  Fort  Ord   [Dilley  and 
Larkins  1973].   Its  primary  resources  are  four  doctors  with 
an  attendant  staff  of  one  nurse  clinician,  several  nurses, 
aides  and  receptionists.   A  patient  population  of  about  4000 
people,  or  1000  families,  was  being  served  by  the  FPC  during 
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the  period  of  the  study,  October  1973  to  April  1974.   In- 
cluded in  this  population  were  active  duty,  dependent  and 
retired  personnel  of  all  ages  and  from  all  branches  of  the 
armed  forces. 

B.   PROBLEM  STATEMENT 

The  staff  of  the  FPC  routinely  had  collected  statistical 
information  about  its  operation  since  opening  in  August  1973. 
In  order  to  become  a  useful  management  tool,  these  statistics 
had  to  be  analyzed  and  summarized.   Thus  the  first  portion 
of  the  problem  was  to  make  a  comprehensive  statistical  anal- 
ysis of  the  FPC  operation  over  a  period  of  five  months. 

An  important  problem  for  the  FPC  administrators  was  de- 
termination of  the  largest  population  size  for  which  a  four 
doctor  clinic  could  provide  quality  care.   It  was  assumed 
that  quality  care  was  being  provided  at  the  time  of  the  com- 
mencement of  this  study  and  that  this  quality  would  not  be 
affected  as  long  as  the  average  amount  of  time  a  physician 
spent  caring  for  a  patient  did  not  decrease  and  the  waiting 
lines  did  not  grow  at  an  ever-increasing  rate.   Since  arbi- 
trary changes  in  population  size  were  totally  impractical, 
the  second  portion  of  the  problem  was  to  determine,  through 
computer  simulation,  an  estimate  of  the  projected  effects  of 
increasing  the  number  of  patients  assigned  to  each  doctor. 

The  military  physicians'  assistant  (PA)  is  a  skilled 
paraprof essional  health  care  provider  whose  responsibility 
level  lies  somewhere  between  a  nurse  clinician  and  a  physi- 
cian.  Although  no  PA's  had  been  assigned  to  the  FPC,  the 
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staff  expressed  a  need  to  know  what  effect  the  introduction 
of  a  PA  would  have  on  the  clinic's  patient  population  capac- 
ity.  Therefore,  the  third  portion  of  the  problem  was  to 
estimate  the  probable  impact  of  a  PA. 
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II.   STATISTICAL  ANALYSIS 

A.   DATA  COLLECTION 

Although  statistics  for  the  FPC  from  August  1973,  the 
clinic's  beginning  month,  through  March  1974  were  available 
for  analysis,  only  October,  November,  January,  February  and 
March  were  considered.   August  and  September  were  not  selected 
because  the  clinic  was  too  young  to  have  reached  a  tempo  of 
operation  approaching  steady  state.   December  was  omitted 
due  to  the  unusually  low  number  of  patient  encounters  during 
the  Christmas  holiday  season. 

The  data  which  were  available  consisted  of  arrival  logs, 
encounter  forms  and  the  appointment  calendar.   Arrival  logs 
were  kept  by  the  receptionists  who  recorded  each  arriving 
patient's  identification,  arrival  time  and  the  nature  of  the 
complaint.   Encounter  forms  were  initiated  by  the  reception- 
ist for  each  patient  and  completed  by  the  staff  members  of 
the  clinic  with  whom  the  patient  interacted  during  his  visit. 
The  most  recent  version  of  the  encounter  form  is  reproduced 
in  Figure  1.   A  blank  page  of  the  appointment  calendar  is 
reproduced  in  Figure  2. 

The  encounter  forms  and  appointment  calendar  pages  were 
collected  and  filed  by  the  statistical  staff  of  the  clinic. 
A  cross  check  of  patient  totals  derived  from  the  encounter 
forms  with  the  totals  derived  from  the  appointment  calendar 
revealed  a  disagreement  of  less  than  0.3%.   Thus  ample  and 
complete  data  were  available  for  the  analysis.   All  of  the 
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data  taken  pertained  only  to  the  principal  resources  of  the 
clinic  —  the  four  physicians  —  and  did  not  include  night- 
time, off-duty  or  weekend  encounters. 

B.  ENCOUNTER  DATA 

The  patient  encounter  data  taken  from  5455  encounter 
forms,  averaged  over  five  months,  for  all  four  doctors,  are 
summarized  in  Table  I. 

In  an  attempt  to  make  certain  that  the  population  of 
patients  allocated  to  each  doctor  mirrored  the  age,  sex  and 
status  distribution  of  the  total  patient  population,  each 
new  family  was  assigned  a  doctor  on  a  random  basis.   Never- 
theless, the  doctors'  panels  were  not  identical  in  size. 
The  five  month  average  number  of  patient  encounters  per 
month  per  doctor,  as  shown  in  Table  II,  was  found  to  be  pro- 
portional to  the  total  number  of  patients  on  the  doctor's 
panel. 

C.  APPOINTMENT  CALENDAR 

During  the  past  ten  years,  the  most  productive  analyses 
of  outpatient  clinic  utilization  and  efficiency  have  con- 
cerned themselves  with  the  clinics'  appointment  calendars. 
An  incontrovertible  conclusion  of  this  work  has  been  that  a 
poorly  designed  appointment  system  inevitably  results  in  re- 
duced physician  utilization  and  increased  patient  waiting 
times  which  contribute  to  patient  dissatisfaction  [Stimson 
and  Stimson  1972].   The  structure  of  the  appointment  calen- 
dar directly  influences  the  number  of  patients  who  are  seen 
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TABLE  I 
ENCOUNTER  DATA  SUMMARY 


ENCOUNTERS  PER  MONTH 


1089 


ENCOUNTERS  PER  WORKDAY 


55 


DAILY  DISTRIBUTION 

(%) 

MONDAY 

23 

TUESDAY 

18 

WEDNESDAY 

20 

THURSDAY 

20 

FRIDAY 

19 

TIME  OF  DAY  (%) 

0800-1200 

63 

1330-1630 

37 

SEX  OF  PATIENT  (%) 

MALE 

38 

FEMALE 

62 

AGE  OF  PATIENT  (%) 

0-12 

26 

13-18 

8 

19-35 

34 

36-50 

20 

OVER  50 

12 

STATUS  (%) 

SPONSOR 

21 

SPOUSE 

44 

DEPENDENT 

35 

APPOINTMENT  STATUS 

(%) 

SCHEDULED 
APPOINTMENT 

92 

WALK- IN 

8 

EMERGENCIES  (%) 

1 

15 


by  a  doctor  on  a  given  day,  the  amount  of  time  the  doctor 
devotes  to  each  patient,  the  waiting  times  of  the  patients 
and  the  idle  time  of  the  doctor. 

TABLE  II 
MONTHLY  DOCTOR  ACTIVITY 

DOCTOR ENCOUNTERS  PER  MONTH %_ 

#1  .                    313  29 

#2                     276  25 

#3                     279  26 

#4                     225  21 

In  addition  to  its  use  for  the  routine  scheduling  of  pa- 
tients, the  calendar  at  the  FPC  was  used  to  record  the  oc- 
currence of  doctor-patient  encounters,  canceled  appointments, 
patients  who  failed  to  appear  for  appointments  and  the  allo- 
cation of  the  doctors'  time  for  other  than  routine  medical 
practice.   Consequently,  this  calendar  information  became 
the  prime  statistical  source  in  the  effort  to  determine  the 
distribution  of  time  allotted  by  the  principal  resource  — 
the  doctors  —  to  various  activities  throughout  a  normal 
working  day. 

A  page  of  the  appointment  calendar  is  shown  in  Figure  2. 
The  fundamental  time  unit  was  15  minutes,  with  appointments 
scheduled  for  15,  30,  45  or  60  minutes.   The  duration  of  an 
appointment  was  determined  by  the  physician  according  to  the 
nature  of  the  patient's  problem.   Although  time  was  allotted 
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in  15  minute  increments,  the  appointments  were  scheduled  to 
begin  on  the  hour,  and  at  10,  30  and  40  minutes  after  the 
hour.   This  scheduling  method  permitted  the  simultaneous  use 
of  two  examining  rooms  per  doctor  to  undress  and  prepare 
patients  before  they  were  actually  seen  by  the  doctor.   A 
maximum  of  28  time  periods,  each  of  15  minutes,  were  avail- 
able every  day.   Appointment  times  shown  with  an  asterisk 
were  withheld  for  acute  illnesses  that  required  a  doctor's 
attention  within  less  than  24  hours.   Thirty-two  percent 
(nine  periods)  of  the  total  number  of  15  minute  time  periods 
in  each  day  were  reserved  in  this  manner  for  semi-emergencies, 

The  distribution  of  time,  in  minutes,  which  all  four 
doctors  devoted  to  various  activities  is  given  in  Table  III. 
The  DOCTOR-PATIENT  column  was  the  time  spent  caring  for  pa- 
tients.  LATE  CANCEL  NO-SHOW  was  unused  time  due  to  appoint- 
ments that  were  canceled  too  late  to  be  rescheduled  for 
another  patient  and  appointments  for  which  patients  simply 
failed  to  appear.   NOT  SCHEDULED  was  time  not  scheduled  ei- 
ther because  patients  were  not  able  to  accept  appointments 
at  available  times  on  the  calendar  or  because  the  number  of 
semi-emergencies  was  insufficient  to  fill  up  the  nine  daily 
time  periods  allotted  for  them.  OTHER  time  was  allocated  for 
annual  leave,  medical  research,  conferences  and  other  bona 
fide  activities. 

The  distribution  of  the  length  of  scheduled  appointments 
for  each  doctor  was  also  derived  from  the  appointment  calen- 
dar.  Table  IV  summarizes  this  data,  averaged  over  five 
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TABLE  IV 
DISTRIBUTION  OF  APPOINTMENTS 


DOCTOR 


15 


MINUTES 
30      45 


60 


AVERAGE 
TIME 


#1 
#2 
#3 
#4 


85% 

14% 

1% 

0% 

17.4  min 

78 

16 

4 

2 

19.5 

76 

17 

5 

2 

20.0 

86 

11 

2 

1 

17.7 

OVERALL 


81 


15 


18.6 


19 


months.   Entries  are  percentages  of  the  total  number  of  each 
doctor's  appointments.   AVERAGE  TIME  is  the  mean  time  per 
scheduled  appointment  in  minutes. 
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III.   SIMULATION  MODEL 

A.   DESCRIPTION 

The  computer  simulation  model  of  the  FPC  was  constructed 
using  the  General  Purpose  Simulation  System  (GPSS)  language. 
A  listing  of  the  program  appears  in  Appendix  A.   The  data 
used  to  build  the  model  were  derived  primarily  from  the  used 
FPC  appointment  calendars.   A  weekly  demand  rate  for  out- 
patient health  care  was  estimated  from  a  five  percent  random 
sample  of  a  one  million  member  civilian  prepaid  health  plan 
during  1971.   In  queueing  theory  terminology,  the  model  con- 
sisted of  four  parallel  M/G/l  queues  for  the  four  doctors 
plus  an  additional  M/G/l  queue  for  the  physicians'  assistant. 
The  statistical  measure  of  effectiveness  of  interest  was  the 
number  of  patients  remaining  in  each  queue  at  the  end  of  each 
week,  and  their  average  over  the  52  weeks  of  a  year. 

The  civilian  health  plan  data  indicated  that  approximate- 
ly four  appointment  requests  per  patient  per  year  could  be 
expected  on  the  average.   However,  the  intensity  of  these 
demands  was  not  constant  over  time.   Figure  3  shows  the  non- 
homogeneous  arrival  pattern  over  one  year  which  was  incor- 
porated in  the  model  after  adjustments  for  the  FPC  population 
size.   Weekly  arrivals  were  assumed  to  be  distributed  Poisson 
with  a  mean  fluctuating  as  in  Figure  3.   The  arrivals  were 
distributed  among  the  four  doctors  in  proportion  to  each  doc- 
tor's alloted  portion  of  the  patient  population.   Thus  the 
four  doctors  received  31%,  24%,  24%  and  21%  of  all  appointment 

requests  respectively. 
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Service  time  was  interpreted  to  be  the  amount  of  appoint- 
ment calendar  time  allocated  to  an  appointment.   The  empiri- 
cal distribution  of  these  times  was  taken  directly  from  the 
appointment  calendar  data. 

The  appointment  calendar  contained  28  fifteen  minute 
periods  each  day  for  five  days  each  week.   Consequently,  the 
simulation  run  length  was  2100  minutes  per  week  or  109200 
minutes  each  year.   The  arrival  rate  was  adjusted  at  the  be- 
ginning of  each  week  to  conform  to  the  rates  depicted  in 
Figure  3.   The  number  of  patients  remaining  in  each  doctor's 
queue  at  the  end  of  each  week  were  printed  in  matrix  form  at 
the  end  of  each  yearly  run.   In  addition,  the  percentage  of 
calendar  time  not  scheduled,  time  allotted  for  appointments 
which  were  kept  and  idle  time  due  to  late  cancellations  and 
no-shows  as  well  as  calendar  utilization  were  also  printed 
in  matrix  form.   Appendix  B  contains  an  annotated  example  of 
the  simulation  output. 

B.   SENSITIVITY  ANALYSIS 

Initial  runs  were  conducted  without  the  physicians'  as- 
sistant in  order  to  validate  the  computer  program  and  to 
conduct  a  sensitivity  analysis  under  current  operating  con- 
sitions.   Table  V   compares  the  calendar  utilization  of  the 
simulation  model  with  the  utilization  derived  from  the  data. 
All  of  the  percentages  of  error  were  less  than  or  equal  to 
1.1%. 
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TABLE 

V 

UTILIZATION 

COMPARISON 

QUEUE 

MODEL 

DATA 

ERROR 

%  ERROR 

Doctor  #1 

92% 

91% 

+1 

1.1% 

Doctor  #2 

87 

87 

0 

0.0 

Doctor  #3 

89 

90 

-1 

1.1 

Doctor  #4 

87 

87 

0 

0.0 

TOTAL  SYSTEM 

89 

89 

0 

0.0 

The  weekly  queue  lengths  displayed  in  the  output  matrix 
were  examined  to  ascertain  the  effects  of  the  non-homogeneous 
patient  arrival  rate.   A  typical  output  matrix  is  shown  in 
Appendix  B.   Queue  lengths  tended  to  increase  to  a  relative 
maximum  during  the  first  twenty  weeks  of  simulation  --  the 
influenza  season  —  and  decreased  to  a  relative  minimum  dur- 
ing the  summer  months.   A  gradual  increase  in  queue  length 
began  again  as  the  winter  months  approached. 

In  order  to  estimate  the  maximum  patient  capacity  of  the 
system,  the  arrival  rate  was  accelerated  until  the  individual 
calendars  of  the  doctors,  and  eventually  the  total  system, 
began  to  form  infinitely  growing  queues.   Table  VI   depicts 
calendar  utilization  as  a  function  of  increased  arrival 
rates  from  a  simulated  ten  year  sample.   As  the  individual 
utilizations  increased  beyond  100%,  the  corresponding  calen- 
dar queues  began  to  grow  at  a  monotonic,  non-decreasing  rate. 
These  overload  points  are  shown  in  the  last  column  of  Table 
VI. 
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TABLE  VI 
CALENDAR  UTILIZATION  AT  INCREASED  ARRIVAL  RATES 


QUEUE 


LOAD   (%  OF  NORMAL) 


100%   104%   110%   114%   118% 


OVERLOAD  POINT 
(%  OF  NORMAL  LOAD) 


#1 
#2 
#3 
#4 

TOTAL 

SYSTEM 


92* 


lO 


87 
89 
87 

89 


94%  100%  100%  100% 

90  95  97  100 

92  97  100  100 

89  95  98  100 


91 


97 


98    100 


110% 
118 
114 
118 

116 


C.   PHYSICIANS'  ASSISTANT 

The  physicians'  assistant  was  introduced  into  the  simula- 
tion model  in  order  to  estimate  his  potential  impact  on  the 
patient  capacity.   It  was  assumed  that  the  PA ' s  calendar 
would  operate  at  a  level  of  utilization  slightly  less  than 
that  of  a  doctor.   Several  trial  runs  using  a  mean  service 
time  of  30  minutes  per  PA  appointment  showed  that  assigning 
14%  of  each  doctor's  patient  arrivals  to  the  PA  resulted  in 
a  PA  utilization  which  was  one  or  two  percent  lower  than  the 
total  system  utilization. 

Addition  of  the  physicians'  assistant  had  a  dampening 
effect  on  the  weekly  and  seasonal  fluctuations  in  queue 
length.   In  addition,  the  system's  total  patient  capacity 
was  increased  substantially.   Arrival  rates  were  accelerated 
to  estimate,  as  before,  when  the  queues  would  begin  to  grow 
monotonically .   The  results  are  given  in  Table  VII.    The 
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point  at  which  the  total  system  queue  began  to  grow  at  an 
ever-increasing  rate  occurred  at  an  arrival  rate  that  was 
23%  greater  than  the  arrival  rate  for  the  same  point  of  the 
system  without  the  PA. 

TABLE  VII 
CALENDAR  UTILIZATION  AT  INCREASED  ARRIVAL  RATES  WITH  PA 


LOAD 

(%  OF 

NORMAL) 

OVERLOAD  POINT 

QUEUE 

(%  OF 

100% 

121% 

128% 

138% 

141% 

NORMAL  LOAD) 

#1 

80% 

97% 

100% 

100% 

100% 

128% 

#2 

75 

90 

96 

100 

100 

132 

#3 

78 

94 

98 

100 

100 

130 

#4 

76 

92 

98 

100 

100 

130 

PA 

72 

89 

93 

98 

100 

139 

TOTAL 

SYSTEM 

76 

92 

97 

99 

100 

139 
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IV.   VARIANCE  REDUCTION 

As  the  utilization  of  the  physicians'  calendar  was  in- 
creased, the  sample  variance  of  the  output  statistic  also 
became  larger.  Table  VIII  shows  how  the  variance  of  the  queue 
length  estimator  increased  as  the  arrival  stream  was  accel- 
erated beyond  the  current  operating  level  of  the  clinic.   In 
an  attempt  to  reduce  this  variance,  two  variance  reduction 
schemes  were  applied.   The  first  of  these  was  the  method  of 
control  variates;  the  second,  antithetic  variates. 

TABLE  VIII 
EXPECTED  QUEUE  LENGTH  VARIANCE 
TEN  1-YEAR  ITERATIONS 


SYSTEM 
PATIENT  DOCTOR  1  DOCTOR  2  DOCTOR  3  DOCTOR  4  PA  SYSTEM  UTIL. 
ARRIVALS  (%) 

0.0     0.4     1.0   76.3 

54.5    12.3   250.4   92.1 

724.5   101.1  2045.4   96.9 


A.   METHOD  OF  CONTROL  VARIATES 

The  method  of  control  variates  involved  the  use  of  a 
second  model  whose  simulation  could  be  positively  correlated 
with  the  original  model  and  for  which  the  mean  value  of  the 
estimator  for  mean  queue  size  could  be  computed  analytically 
[Gaver  and  Thompson  1973].   A  simple  theoretical  model  which 
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BASE 

0.8 

0.3 

0.9 

+21% 

152.9 

15.3 

115.4 

+28% 

1741.1 

104.4 

183.0 

closely  approximated  the  results  of  the  original  model  had 
to  be  constructed.   The  equation  for  the  queue  size  estima- 
tor was: 

Q  =  E(A)  +  Q  -  A 
where 

E(A)  =  known  expected  queue  length  for  the  control 

model. 
Q    =  mean  queue  length  empirically  found  with  the 

simulation  model. 
A    =  mean  queue  length  empirically  found  with  the 
control  model. 

The  crucial  point  of  this  method  was  that  identical  se- 
quences of  pseudo-random  numbers  had  to  be  used  in  the  simu- 
lations to  compute  Q  and  A  in  order  to  ensure  a  positive 
correlation  between  these  two  statistics.   Q  was  an  unbiased 
estimator  since  E(A)  =  E(A),  and  its  variance  was: 

V(Q)  =  V(Q  -  A)  =  V(Q)  +  V(A)  -2  Cov(Q,A) 

A  positive  correlation  between  Q  and  A  implies  a  positive 
covariance.   This  in  turn  should  decrease  the  variance  of 
Q  to  a  value  less  than  the  variance  which  could  be  realized, 
namely  V(Q)  using  the  ordinary  Monte  Carlo  method,  provided 
that  2  Cov(Q,A)  was  greater  than  V(A). 

The  control  model  used  to  approximate  the  simulation  mod- 
el of  the  clinic  was  a  system  of  five  parallel  M/M/l  queues 
operating  at  the  same  average  activity  level  as  the  original 
model  of  the  clinic.   None  of  the  refinements  such  as  unused 
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time  due  to  no-shows,  late  cancellations  nor  a  time  dependent 
arrival  stream  were  included.   The  system's  level  of  utili- 
zation chosen  for  this  experiment  was  92%  which  implied  a 
patient  arrival  stream  21%  above  normal.   Twenty  Monte  Carlo 
iterations  of  the  original  model  were  compared  with  ten  iter- 
ations of  the  control  variate  method  involving  approximately 
the  same  amount  of  effort  and  total  computer  time.   The  re- 
sults are  shown  in  Table  IX.  Clearly,  the  desired  variance 
reduction  was  not  achieved.   Reasons  for  this  outcome  are 
addressed  in  the  section  entitled  RESULTS  OF  VARIANCE  REDUC- 
TION. 

TABLE  IX 

VARIANCE  OF  QUEUE  LENGTH  ESTIMATOR 
METHOD  OF  CONTROL  VARIATES 


MONTE  CARLO        CONTROL  VARIATE 
yUlLUii  20  ITERATIONS        10  ITERATIONS 


749.2 
16.5 

174.9 
80.0 
18.1 

642.8 


Doctor  1 

119.6 

Doctor  2 

15.5 

Doctor  3 

89.4 

Doctor  4 

33.3 

PA 

13.2 

TOTAL  SYSTEM 

216.0 
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B.   METHOD  OF  ANTITHETIC  VARIATES 

The  method  of  antithetic  variates  involved  the  use  of  a 
second  queue  length  estimator  calculated  from  antithetic 
versions  of  the  original  random  number  stream.   Its  expected 
value  was  the  same  as  that  of  the  original  estimator  and  its 
correlation  with  the  original  estimator  should  be  negative 
[Hammersley  and  Handscomb  1964].   Negative  correlation  was 
achieved  by  using  two  sets  of  antithetically  generated  ran- 
dom numbers  for  two  companion  simulation  runs  respectively. 
That  is,  a  set  of  random  numbers  R,  distributed  uniformly  on 
the  interval  [0,1],  was  used  for  ten  one-year  iterations  of 
the  simulation  model.   Then  a  second  set  of  ten  iterations 
of  the  same  model  was  run  using  the  set  of  random  numbers 
1-R. 

The  antithetic  estimators  were: 

Q  =  Q  +J 

H  2 

where 

Q  =  sample  average  queue  length  using  the  set  R. 
A  =  sample  average  queue  length  using  the  set  1-R. 
The  estimator  is  unbiased  since: 

E(Q)  =  *[E(Q)  +  E(A)]  =  i[E(Q)  +  E(Q)] 
Its  variance  is  reduced  since: 

V(Q)  -  V[J(Q+A)]  =  i[V(Q)  +  V(A)  +  2  Cov(Q,A)] 

and  Cov(Q.A)  <  0. 
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If  the  relationship  between  the  random  number  R  and  Q  as 
well  as  between  1-R  and  A  was  monotonic,  then  the  negative 
correlation  between  1-R  and  R  would  result  in  a  negative 
covariance  between  Q  and  A.   This,  in  turn,  would  result  in 
a  reduction  of  the  variance  of  Q  to  a  value  less  than  that 
which  could  be  expected  using  the  ordinary  Monte  Carlo  meth- 
od. 

Two  antithetically  generated  companion  runs  of  ten  one- 
year  iterations,  in  which  antithetics  were  applied  to  the 
generation  of  inter-arrival  times  as  well  as  service  times, 
were  compared  with  20  iterations  of  the  Monte  Carlo  method, 
involving  approximately  the  same  amount  of  effort  and  compu- 
ter time.   System  level  of  utilization  was  92%.   The  result- 
ant variance  reductions  are  shown  in  Table  X. 

TABLE  X 

VARIANCE  OF  QUEUE  LENGTH  ESTIMATOR 
METHOD  OF  ANTITHETIC  VARIATES 


QUEUE 


MONTE  CARLO 
20  ITERATIONS 


ANTITHETIC 
10  ITERATIONS 


%  REDUCTION 


Doctor 

1 

119.6 

Doctor 

2 

15.5 

Doctor 

3 

89.4 

Doctor 

4 

33.3 

PA 

13.2 

TOTAL  ! 

SYSTEM 

216.0 

77.0 
7.7 

50.4 

21.7 
3.1 

86.7 


36 
50 
44 
35 
77 
60 
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A  second  set  of  antithetic  runs  of  ten  iterations  in 
which  antithetics  were  applied  only  to  the  generation  of  in- 
ter-arrival times  resulted  in  the  variance  reductions  shown 
in  Table  XI. 

TABLE  XI 
ESTIMATOR  VARIANCE 
ANTITHETIC  ARRIVALS 


QUEUE 


MONTE  CARLO 
20  ITERATIONS 


ANTITHETIC 
10  ITERATIONS 


%  REDUCTION 


Doctor  1 

119.6 

Doctor  2 

15.5 

Doctor  3 

89.4 

Doctor  4 

33.3 

PA 

13.2 

TOTAL  SYSTEM 

216.0 

88.5 

7.7 

28.7 

14.3 

4.8 

115.8 


26 
50 
68 
57 
64 
46 


The  results  of  a  final  set  of  runs  in  which  antithetics 
were  applied  only  to  the  generation  of  appointment  times  are 
shown  in  Table  XII. 
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TABLE  XII 
ESTIMATOR  VARIANCE 
ANTITHETIC  APPOINTMENT  TIMES 


QUEUE 


MONTE  CARLO 
20  ITERATIONS 


ANTITHETIC 
10  ITERATIONS 


%  REDUCTION 


Doctor  1 

119.6 

Doctor  2 

15.5 

Doctor  3 

89.4 

Doctor  4 

33.3 

PA 

13.2 

TOTAL  SYSTEM 

216.0 

20.4 
10.6 
37.6 
16.8 
1.6 
63.9 


83 
32 
58 
50 
88 
70 
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V.   SUMMARY  OF  RESULTS 

A.  RESULTS  OF  STATISTICAL  ANALYSIS 

Assuming  that  the  doctor  "OUT"  time  was  fixed  at  a  month- 
ly average  of  8580  minutes,  as  shown  in  Table  III,  the  re- 
sults of  the  statistical  analysis  indicated  that  a  limited 
expansion  of  the  clinic's  patient  capacity  was  possible. 
Regardless  of  the  configuration  of  the  appointment  calendar, 
the  late  cancel/no-show  rate  of  9.5%  of  the  doctors'  "IN" 
time  would  be  difficult  to  change.   However,  the  11.6%  (2937 
minutes  per  month)  of  unscheduled  time  was  available  for 
additional  appointments.   Dividing  2937  minutes  by  the  over- 
all average  service  time  of  18.6  minutes  per  appointment 
yielded  a  maximum  additional  number  of  158  appointments  per 
month.   This  represented  a  theoretical  upper  bound  equal  to 
115%  of  current  operating  conditions.   Operating  the  clinic 
at  this  level  would  be  equivalent  to  operating  at  a  calendar 
utilization  equal  to  100%. 

B.  RESULTS  OF  SENSITIVITY  ANALYSIS 

At  a  simulated  patient  load  between  114%  and  118%  of  the 
clinic's  current  operating  point(  without  the  PA),  all  of 
the  queues  began  to  grow  at  ever-increasing  rates,  all  un- 
scheduled time  became  zero  and  the  utilization  of  all  the 
queues  had  reached  100%.   This  result  was  in  agreement  with 
the  results  of  the  statistical  analysis.   However,  this  theo- 
retical upper  bound  was  not  feasible.   The  output  matrix 
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which  showed  the  weekly  queue  length  fluctuations  over  the 
entire  year,  indicated  that  excessive  queue  lengths  during 
the  early  months  of  the  year  were  realized  well  before  100% 
system  utilization  occurred. 

Discussions  with  the  FPC  staff  had  led  to  the  assumption 
that  a  queue  length  greater  than  about  500  patients  for  the 
system,  or  125  per  doctor,  would  result  in  excessive  waiting 
times  for  the  patients,  implying  a  decrease  in  the  quality 
of  care  as  originally  defined  in  the  problem  statement.   Ac- 
celeration of  the  simulation  model's  patient  arrival  rate 
showed  that  peak  system  queue  lengths  during  May  began  to 
exceed  500  patients  when  the  arrival  rate  was  10%  higher 
than  normal,  at  a  system  utilization  of  97%.   Table  XIII 
shows  the  weekly  average  queue  lengths  of  the  total  clinic 
(without  PA)  for  52  weeks  measured  over  ten  one-year  samples 
at  an  arrival  rate  10%  above  normal.   During  week  20,  the 
average  queue  length  peaked  at  501.4. 

The  second  series  of  simulation  runs,  which  included  the 
PA,  indicated  that  utilization  of  all  the  queues  had  reached 
100%  between  38%  and  41%  above  normal  capacity.   As  in  the 
case  without  the  PA,  the  weekly  fluctuations  of  queue  length 
as  depicted  in  the  output  matrix  led  to  a  reduced  maximum 
patient  capacity  that  was  29%  above  normal.   Table  XIV  shows 
the  weekly  average  queue  lengths  of  the  total  clinic  for  52 
weeks  measured  over  ten  one-year  samples  at  an  arrival  rate 
29%  above  normal  and  a  total  system  utilization  of  96%.   The 
maximum  acceptable  number  of  patients  in  the  calendar  queue 
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TABLE  XIII 
AVERAGE  WEEKLY  QUEUE  LENGTH  WITHOUT  PA 

Week     Queue  Length  Week     Queue  Length 

1  13.5 

2  75.7 

3  61.4 

4  109.8 

5  149.9 

6  190.3 

7  129.2 

8  176.6 

9  219.2 

10  261.3 

11  307.0 

12  355.6 

13  404.3 

14  377.5 

15  356.1 

16  382.7 

17  441.5 

18  410.5 

19  460.8 

20  501.4 

21  475.3 

22  392.0 

23  369.4 

24  344.3 

25  317.3 

26  288.7 


27 

197.8 

28 

188.8 

29 

180.1 

30 

163.4 

31 

154.4 

32 

142.1 

33 

134.5 

34 

122.0 

35 

119.5 

36 

74.4 

37 

80.4 

38 

76.9 

39 

74.8 

40 

116.8 

41 

101.7 

42 

92.7 

43 

78.2 

44 

111.2 

45 

.   165.6 

46 

201.7 

47 

133.4 

48 

113.3 

49 

163.3 

50 

203.4 

51 

169.9 

52 

111.7 
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TABLE  XIV 
AVERAGE  WEEKLY  QUEUE  LENGTH  WITH  PA 

Week     Queue  Length  Week     Queue  Length 

27  .  260.4 

28  248.8 

29  234.4 

30  221.4 

31  215.2 

32  211.6 

33  196.2 

34  171.0 

35  152.0 

36  99.4 

37  99.4 

38  95.2 

39  109.2 

40  168.0 

41  143.2 

42  131.6 

43  129.6 

44  188.0 

45  269.6 

46  349.4 

47  238.6 

48  202.4 

49  275.0 

50  365.8 

51  315.6 

52  221.8 


1 

5.0 

2 

84.4 

3 

50.2 

4 

131.0 

5 

187.2 

6 

247.6 

7 

141.2 

8 

207.6 

9 

285.2 

10 

366.8 

11 

403.2 

12 

468.6 

13 

536.8 

14 

489.2 

15 

456.4 

16 

525.2 

17 

590.0 

18 

543.4 

19 

592.8 

20 

645.8 

21 

588.2 

22 

497.0 

23 

464.0 

24 

439.0 

25 

404.2 

26 

376.8 
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had  been  increased  to  625  to  account  for  the  addition  of  the 
PA.   The  system  average  queue  length  reached  a  maximum  of 
645.8  during  the  twentieth  week. 

Thus,  the  simulation  results  revealed  a  phenomenon  that 
could  not  have  been  quantified  analytically;  namely,  that 
increasing  the  patient  capacity  was  limited  by  the  time  de- 
pendent fluctuations  of  queue  length. 

C.    RESULTS  OF  VARIANCE  REDUCTION  TECHNIQUES 
1.   Control  Variates 

Variance  reduction,  using  the  method  of  control 
variates,  was  not  realized.   The  most  probable  cause  of  this 
method's  vailure  was  a  lack  of  correlation  between  the  main 
simulation  model  and  the  control  model.   To  check  correla- 
tion between  the  two  models,  the  weekly  queue  lengths  of  the 
two  models  were  compared  over  the  period  of  one  year  of  sim- 
ulation.  The  results  of  this  comparison  are  shown  in  Table 
XV.    Only  the  total  system  queues  are  shown.   However,  doc- 
tor queue  length  differences  between  the  two  models  were  of 
approximately  the  same  magnitude  as  total  system  queue  length 
differences. 

The  sample  correlation  coefficient  (rn)  was  calcu- 
lated using  the  following  formulas: 

Sl  2 

1-12   = 


SiS2 

-   52 

Sl2   =  51  k^l   ^k  "  X!)(X2k  "  V 
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TABLE  XV 
WEEKLY  SYSTEM  QUEUE  LENGTH 


T17   .   Main   Control  r.-** 
Week  ,,  ,  ,   ,,  ,  ,   Difference 
Model   Model 


(1)     (2) 


(1  -  2) 


„.   ,   Mam   Control  ^-^^ 
Week  ,,  ,  ,   ,.  ,  ,   Difference 
Model   Model 


(1).     (2) 


(1  -  2) 


1 

15 

135 

-120 

27 

5 

128 

-123 

2 

52 

124 

-  72 

28 

11 

107 

-  96 

3 

35 

92 

-  57 

29 

16 

110 

-  94 

4 

76 

96 

-  20 

30 

20 

145 

-125 

5 

90 

66 

24 

31 

22 

145 

-123 

6 

94 

55 

39 

32 

13 

190 

-177 

7 

36 

51 

-  15 

33 

33 

162 

-129 

8 

53 

56 

-   3 

34 

12 

158 

-146 

9 

85 

38 

47 

35 

19 

124 

-105 

10 

96 

77 

19 

36 

2 

151 

-149 

11 

136 

67 

69 

37 

15 

161 

-146 

12 

159 

98 

61 

38 

26 

192 

-166 

13 

185 

86 

99 

39 

25 

181 

-156 

14 

159 

69 

90 

40 

41 

164 

-123 

15 

134 

63 

71 

41 

24 

146 

-122 

16 

155 

96 

59 

42 

24 

143 

-119 

17 

207 

86 

121 

43 

3 

139 

-136 

18 

181 

68 

113 

44 

74 

136 

-  62 

19 

226 

89 

137 

45 

99 

126 

-  27 

20 

268 

102 

166 

46 

110 

72 

38 

21 

198 

115 

83 

47 

44 

61 

-  17 

22 

81 

133 

-  52 

48 

40 

44 

-   4 

23 

66 

105 

-  39 

49 

55 

42 

13 

24 

35 

117 

-  82 

50 

81 

43 

38 

25 

28 

150 

-122 

51 

34 

29 

5 

26 

31 

135 

-104 

52 

24 

43 

-  19 

39 


1   52  * 

S.  =  [-gr        Z         (X.,   "  X.)2]* 

i    51  ^=1    ik    1 


j   52 
Xi  =  52  k£i  xik 


1  MAIN  MODEL 
i  = 

2  CONTROL  MODEL 


x 


•k  =  queue  length  of  column  i,  week  k, 


For  the  two  model  runs  of  52  weeks  depicted  in  Table 
XV,  the  sample  correlation  coefficient  was  -0.38,  indicating 
that  the  correlation  between  the  two  models  was  not  strong. 

The  lack  of  correlation  was  attributable,  in  part, 
to  the  fact  that  the  patient  arrival  stream  of  the  control 
model  was  not  time  dependent,  whereas  it  was  in  the  main 
model.   The  weekly  queue  length  of  the  main  model  rose  to  a 
peak  of  268  during  week  20,  while  the  queue  length  peak  of 
192  of  the  control  model  occurred  during  week  38.   A  compar- 
ison check  of  nine  additional  52-week  runs  showed  that  the 
relative  positions  of  the  queue  length  peaks  of  the  two  mod- 
els did  not  change  significantly.   Therefore,  making  the 
control  model  patient  arrival  stream  non-homogeneous  with 
respect  to  time  was  investigated. 

The  control  model  required  220  K  bytes  of  computer 
core  and  about  20  minutes  of  total  computer  time  per  ten- 
year  run.   Although  it  was  much  more  complex,  the  original 
model  also  required  220  K  bytes  and  approximately  55  minutes 
by  comparison.   Introducing  the  time  dependence  into  the 
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control  model  involved  a  large  Increase  in  its  complexity 
and  lengthening  its  run  time  to  about  50  minutes.   Moreover, 
a  simple  analytical  solution  for  the  expected  queue  length 
of  the  control  model  was  no  longer  obvious. 

Finally,  to  ensure  that  the  patient  arrivals  of 
each  model  were  in  complete  synchronization,  the  two  models 
had  to  be  combined  into  one  computer  program  in  which  each 
patient  arrival  was  split  into  two  arrivals,  one  per  model, 
having  identical  arrival  times. 

The  sum  total  of  these  changes  resulted  in  a  very 
complicated  program  requiring  approximately  440  K  bytes  of 
core  and  a  run  time  of  about  100  minutes,  without  a  known 
expected  value  for  the  control  model  output  statistic.   Hence, 
the  possibly  variance  reduction  advantages  were  completely 
overshadowed  by  the  new  program's  enormity,  and  this  ap- 
proach was  abandoned. 

2.   Antithetic  Variates 

The  success  of  the  method  of  antithetic  variates  in 
reducing  the  variance  of  the  queue  length  statistic  was  due 
primarily  to  the  monotonic  relationship  between  the  random 
numbers  used  to  generate  inter-arrival  and  service  times  and 
the  output  statistic.   It  was  reasonable  to  expect  that  a 
large  random  number  would  result  in  a  large  inter-arrival 
time  and  subsequently  a  shorter  queue  length  as  the  result 
of  a  decreased  arrival  rate.   Conversely,  a  small  random 
number  would  result  in  a  short  inter-arrival  time,  or  higher 
arrival  rate,  and  subsequently  a  longer  queue  length. 
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However,  the  expected  antithetic  effects  were  entirely  oppo- 
site for  the  generation  of  service  times.   That  is,  large 
random  numbers  would  result  in  longer  service  times  and 
longer  queues.   Whether  or  not  the  inter-arrival  and  service 
time  effects  would  tend  to  cancel  each  other  was  not  entirely 
clear  at  the  beginning  of  the  variance  reduction  experiments. 

The  results,  as  shown  in  Tables  X,  XI,  and  XII  in- 
dicated that  whether  antithetics  were  applied  to  appointment 
arrivals,  appointment  times  or  both  did  not  have  a  signfi- 
cant  effect  on  the  variance  reductions  realized.   All  three 
antithetic  methods  were  approximately  equal  in  effect. 
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VI.   CONCLUSIONS  AND  RECOMMENDATIONS 

The  following  statements  explain  the  point  of  view  sole- 
ly of  the  author  and  should  not  be  construed  as  a  reflection 
of  the  opinions  of  members  of  the  FPC  staff. 

The  statistical  analysis  showed  that  8.7%  of  the  doctor's 
total  calendar  time  was  unscheduled.   More  than  half  of  this 
amount  were  time  periods  that  had  been  withheld  for  semi- 
emergencies.   Consequently,  a  14%  reduction  of  the  time  al- 
located to  semi-emergencies  could  result  in  an  increase  of 
4.4%  (1469  minutes)  in  time  available  for  scheduling  routine 
appointments.   This  converts  into  77  additional  patient  en- 
counters per  month,  an  increase  of  7%.   As  it  involves  no 
changes  in  configuration  or  the  total  time  the  staff  spends 
at  the  clinic,  this  relatively  simple  change  is  recommended. 

A  second  change  to  increase  patient  capacity  would  be  to 
lengthen  the  doctors'  daily  working  hours  from  seven  to 
eight.   An  additional  5040  minutes  per  month  would  be  avail- 
able for  all  activities.   After  subtracting  7.1%  for  no- 
shows  and  late  cancellations,  8.7%  for  unscheduled  time  and 
25.3%  for  OTHER  activities,  2969  minutes  remain  for  addi- 
tional patient  care  each  month.   This  translates  into  156 
more  patient  encounters  per  month,  an  increase  of  14%. 
Since  implementation  of  this  change  implies  a  change  in  clin- 
ic operating  policy,  it  is  presented  for  possible  considera- 
tion only. 
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A  third  change,  resulting  in  increased  capacity,  would 
be  to  reduce  the  percentage  of  time  devoted  to  "OTHER"  bona 
fide  activities  (25.3%).   How  this  could  be  accomplished  is 
beyond  the  purview  of  this  study.   A  previous  study  of  an 
outpatient  clinic  indicates  that  allocating  only  15%  to  this 
activity  would  be  feasible  [Fetter  and  Thompson  1965].   A 
reduction  of  OTHER  time  from  25.3%  to  15%  translates  into 
148  more  patients  per  month  after  time  for  no-shows  and  late 
cancellations  and  unscheduled  time  have  been  subtracted. 
This  represents  a  14%  potential  increase  in  the  capacity  of 
the  doctors'  patient  population. 

Results  of  the  sensitivity  analysis  indicated  that  the 
most  significant  potential  increase  in  patient  capacity 
could  be  achieved  by  the  addition  of  a  physicians'  assistant 
to  the  staff.   As  was  shown  in  the  section  entitled  PHYSI- 
CIANS' ASSISTANT,  the  desired  PA  utilization  of  one  or  two 
percentage  points  less  than  the  overall  system  utilization 
could  be  realized  by  assigning  14%  of  each  doctor's  appoint- 
ments to  the  PA.   Therefore,  without  changing  the  percentages 
of  time  currently  allocated  as  in  Table  III,  a  concomitant 
increase  of  14%  in  the  number  of  appointments  would  be  pos- 
sible.  Coupled  with  the  least  effective  change  involving 
reduction  of  semi-emergency  time,  a  potential  21%  increase 
in  patient  appointments  per  month  could  be  achieved.   This 
alternative  is  highly  recommended. 

Considering  the  total  amount  of  time  and  effort  devoted 
to  the  variance  reduction  experiments,  the  results  were 
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marginally  effective  at  best.   The  control  variate  method 
should  not  be  attempted  unless  a  simple  theoretical  control 
model,  with  known  expected  values  and  a  strong  correlation 
to  the  original  model,  is  available.   Significant  departures 
from  the  simple  control  model  render  the  method  inefficient 
in  comparison  with  the  method  of  antithetic  variates.   Chang- 
ing a  handful  of  computer  cards  was  the  only  work  involved 
in  employing  antithetics.   The  fact  that  approximately  equal 
variance  reductions  were  achieved  whether  antithetics  were 
applied  to  arrivals  only,  appointment  lengths  only,  or  to 
both  is  an  interesting  result. 
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1st  Matrix 

Columns 

1 

Doctor  1  queue 

2 

Doctor  2  queue 

3 

Doctor  3  queue 

4 

Doctor  4  queue 
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Total  System  queue 
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2nd  Matrix 

%  of  total  time  caring  for  patients 

%  of  total  time  for  no-shows,  late  cancellations 

%  of  total  time  not  scheduled 

Doctor  utilization  (%) 

Calendar  utilization  (%) 
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PA 
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